Semi-supervised Semantic Role Labeling

Most corpus-based approaches to language learning have focused on tasks for which a sufficient amount of human-labeled training data is available. However, it is difficult to produce such data, and models trained from such data tend to be brittle when applied to domains that vary, even in seemingly minor ways, from the training data. We claim that these difficulties can by overcome by applying semi-supervised learning techniques. Semi-supervised techniques learn from both labeled and “raw” data. In our case, the latter is raw text. Several researchers have used semi-supervised techniques for language learning (Nigam et al. 2000; Blum & Mitchell 1998; Joachims 1999; Riloff & Jones 1999), but we believe that this area is not yet well explored and definitely not well understood. Therefore, we present a challenge problem for semi-supervised learning: semantic role labeling and semantic relationship annotation. Semantic role labeling was introduced by Gildea & Jurafsky (2002), and we added semantic relationship annotation in Thompson, Levy, & Manning (2003). This problem is a difficult one for semisupervised techniques, for three reasons. First, there are many possible classes (the role labels) for examples. Second, sequence learning is involved. Third, the learning scenario is plagued by sparse data problems. We describe the role labeling problem, our learning model and its extendibility to semi-supervised learning, and some preliminary experiments. Semantic Role Labeling Extracting semantic meaning from the surface form of sentences is important for many understanding tasks performed by human and machine alike. These include inference, antecedent resolution, word sense disambiguation, or even syntactic tasks such as prepositional phrase attachment. We focus here on a particular type of semantic analysis, that of determining the semantic roles and relationships at play in a sentence. For example, in the sentence ∗Some of this work was supported by an ARDA Aquaint grant while the author was a Visiting Assistant Professor at Stanford University. Carolin Arnold at the University of Utah was crucial in running the EM experiments. Copyright c © 2004, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. “The hammer broke the vase,” we may want to find the participants, or roles, in the breaking action. In this case the roles are instrument, filled by “the hammer,” and patient, filled by “the vase.” We may also care about the over-arching relationship created by these roles and the breaking action, which by the theory we are using is the CAUSE TO FRAGMENT relationship. Having semantic roles allows one to recognize the semantic arguments of a situation, even when expressed in different syntactic configurations. For example, in “I broke the vase with the hammer,” the vase and hammer play the same roles as in the previous sentence. We have developed a generative model for performing this type of semantic analysis. Generative models are probability models representing a joint distribution over a set of variables. The specific probability settings are called parameters. As pointed out by Jurafsky (2003) and others, probabilistic models capture cognitively plausible aspects of human language processing, generation, and learning. Our model takes as input the constituents of a sentence and a predicator word (or phrase) from that sentence. In the previous example broke is the predicator. The predicator takes semantic role arguments, instantiated by the constituents. We learn the parameters for this model from a body of examples provided by the FrameNet corpus (Baker, Fillmore, & Lowe 1998). The problem and some elements of our approach are similar to that of Gildea & Jurafsky (2002), but our work differs by use of a generative, not a discriminative (conditional), model. We also add the inference of the overarching relationship between the roles, called the frame.